Missouri Course Redesign Initiative

University of Missouri-St. Louis

Course Title: Computers and Information Systems
Redesign Coordinator: Mimi Duncan

Project Abstract
Final Report (as of 3/1/13)

Project Abstract

Computers and Information Systems is the only large enrollment course (~550 students per year) in the College of Business Administration at the University of Missouri-St. Louis (UMSL) that meets a general education requirement. In addition, it is a required course for all business majors. The skills students acquire are regarded as foundational: students’ success in this course results in the technological literacy necessary to be successful in future courses and in the workplace. These literacy skills align with the university’s strategic direction in its mission and its Gateway to Greatness strategic plan.  The traditional course structure is a three-hour lecture-based course taught in a computer classroom with by a variety of full- and part-time faculty, with the support of four graduate teaching assistants (GTAs) who work as tutors.

The course faces persistent technological challenges because the course focus and resources must constantly adapt to upgrades in operating systems and software applications. Such frequent technology improvements require that all course instructors develop corresponding expertise within a short time period. When part-time faculty are reluctant to take the time to update skills, course drift results. Analysis of the DFW rates demonstrates that students are not as successful as expected. This trend is especially apparent in evening courses. The evening sections serve a very non-traditional student population and have the highest failure rate (ranging from 33–51%). UMSL’s history of serving the non-traditional student is a mission-driven commitment to the community that is noted throughout the university’s strategic plan. When students do not successfully complete Computers and Information Systems, they may lack the fundamental skills needed to be successful in upper-level courses. While the College of Business Administration has a large number of majors, the department of information systems seeks to increase its number of majors and graduate students. 

In the redesign, one instructor will be the instructor of record for one large course section. The course will utilize commercially available instructional software and enlist a mix of undergraduate peer tutors and GTAs to assist with course communications and grading. The instructor will identify learning outcomes, design course activities and assessments, coordinate the delivery of the course content and consolidate a common set of scoring guides to be used by the GTAs. Students will be required to attend the initial class meeting of the semester to learn the course expectations and prepare for the active learning environment. The balance of the scheduled class meetings will take place in smaller technology classrooms and will require successful completion of automated tasks to check performance on and mastery of the assigned modules. The instructor and undergraduate peer tutors will be available at workstations among the students to respond to questions and guide instruction. 

The move to the Replacement Model is an important next step in UMSL’s commitment to improved student learning. Collapsing all sections into one, increasing the number of active learning opportunities, embedding tutors more deeply into the course and utilizing more intimate learning opportunities in smaller venues will help ensure that students spend more time on task and have more meaningful opportunities to apply the course theories and principles.

During the pilot and full implementation phases of the redesign, the team will compare performance data on a common final exam from prior semesters with student performance after the redesign. In addition to comparing students’ performance on the common final exam, they will develop common rubrics to analyze students’ work throughout the semester. When data on a particular assignment are aggregated, the team will be able to analyze instructional activities for their effectiveness and adapt them as necessary to meet course objectives. 

The redesign of Computers and Information Systems will result in a decrease in the cost per student from $113 to $95, a 16% reduction. This cost savings is a result of consolidating all sections under one instructor assisted by a team of undergraduate and graduate students. The number of sections will decrease from 11 to two annually and section size will increase from 50 to ~150-200 students. The cost savings will be allocated back to the information systems department within the College of Business Administration for student recruitment and outreach and other departmental needs.

 

Final Report (as of 3/1/13)

Impact on Students

In the redesign, did students learn more, less or the same compared to the traditional format?

Improved Learning

Student learning outcomes were compared by administering the same final exam to both groups. In the traditional course, final exam scores averaged 73%, while in the redesigned course final exam scores averaged 79%.

For Pell-eligible students, the average final exam score was 69% in the traditional course and 77% in the redesigned course. 

Improved Completion

There was no significant difference in the final grades for the course: 85% of students in the traditional course received a passing grade (defined as C or better) versus 84% of students in the redesigned course.

It is important to remember that the baseline of the traditional course was taught by several different instructors and the only constant in all those sections was the final exam. The adjuncts who taught in fall 2011 were aware that this was probably their second-to-last semester teaching the course, so some grade inflation may have occurred in the traditional course.

Other Impacts on Students

UMSL conducted a study that showed the redesign students did better in the subsequent Information Systems class than students from the traditional course.

Impact on Cost Savings

Were costs reduced as planned?

UMSL carried out its original cost savings plan, which was to consolidate all sections under one instructor assisted by a team of undergraduate and graduate students. The number of sections would decrease from 11 to two annually and section size would increase from 50 to ~150-300 students. The planned enrollment was 550 students. These changes would have resulted in a decrease in the cost-per-student from $113 to $95, a 16% reduction.

However, the actual enrollment increased to 640, yielding even greater savings.  The cost-per-student decreased from $113 in the traditional sections to $81 in the redesign, a 28% reduction.  

Lessons Learned

Pedagogical Improvement Techniques

What techniques contributed most to improving the quality of student learning?

Active learning. The integration of an active learning environment supported by online resources contributed to student success on collaborative and individual student projects. Students could demonstrate mastery of course concepts and apply what they were learning within the business context. Redesign students were given the tools in the applications but had to take responsibility for their own learning and learn to seek help when needed. They had access to multiple technology-laden resources, and they achieved the competencies in a manner that would aid knowledge transfer as they continued at the institution. The course syllabus included language to prepare students for this new environment and emphasized the importance of seeking help and managing their own learning.

Extensive use of learning management system. The foundation of the redesigned course was the Blackboard learning management system. All content and assessments were delivered here. In addition, weekly updates on class events kept the students informed. With one source for all information, the students never had to hunt for content or resources and could spend time on skill acquisition.

Custom portal by the publisher. The textbook provided a framework and supplemental exercises that complemented the examples, assignments and guided lectures created by the instructor. The seamless integration of both publisher and instructor content proved beneficial to students to create a one-stop shop for all coursework: files, practice tests, online glossary, texts, assignments and assessments.

Low stakes quizzes and blogs. During weeks when no exams were given, the students were assigned five-point quizzes or reflective blog entries to keep them engaged in the course and prompting them to think about what they were learning. Not only did the multiple-choice quizzes reinforce topics to be discussed, but the students learned to create blog posts on relevant technology topics that were based on current news items. Analysis demonstrated that the students who participated in these ten opportunities fared better in the class. Looking at all 316 students, 172 earned all the points available for the low stakes quizzes and blogs. The average grade for these students was an A, which indicated a correlation between participation and high success.

Embedded support. Weekly student support labs where students could receive just-in-time answers while working on projectsincreased the learning resources for all students. The labs were staffed by the instructor, graduate learning assistants and an undergraduate learning assistant (ULA). In addition, the ULA attended many class meetings to provide help in real time. The casual nature of these support labs provided an atmosphere in which students felt comfortable asking for help on projects. In previous years, they may have been reluctant to interrupt and ask a question. In the traditional course, students rarely attended office hours and underutilized tutors for this help. The individualized attention as the students needed the help contributed to the improved average of scores on the final exam for all students.

Cost Reduction Techniques

What techniques contributed most to reducing costs?

Reduction in the number of instructors.  One full-time instructor taught all students. Adjuncts (3-4 per semester) were no longer required.

Change in the mix of personnel from more expensive to less expensive. Prior to the redesign, the four tutors dedicated to the course were underutilized and not embedded in the course. Rather, they held office hours for students to drop in as needed, which students rarely did. In the redesign, these tutors joined the class in the support labs to be immediately available as questions arose. This support for students was key to also supporting the instructor who in the redesign has a marked increase of responsibility. These graduate learning assistants also helped grade and monitor student participation in an assigned subsection of the total student population. The addition of an embedded ULA added a low-cost support for students, as the ULA recently had successfully completed the course and knew where the trouble spots were.

Implementation Issues

What implementation issues were most important?

The value of prior experience. Because the course had been redesigned for two delivery modes (traditional and online) prior to this redesign, the blend of the best of both positioned the implementation of the replacement model as a natural progression. This prior experience helped the redesign team hit the ground running.

Scheduling issues. Combining multiple sections into one large enrollment section proved challenging for UMSL’s classroom management and scheduling system. Unfortunately, the university does not have a technology classroom large enough to accommodate more than 54 students at one time. Six recitation sessions were held during the week on two different days including an evening session to accommodate non-traditional students. Until the university renovates a building to provide a 300-seat collaborative space, with or without permanent computers, the silver lining of students’ perceptions of a small learning community within the larger enrollment course can still be enjoyed.

Administration of the final exam. The original redesign plan for assessments required students to take exams in the testing center. As the full implementation began, information technology services had to close one of the two testing centers on campus due to budget cuts, which resulted in a campus-wide increase in demand for the remaining testing center. As the first exam approached, many students were unable to schedule or take the exam at the testing center, so additional class meetings were required to satisfy that demand.  As the semester continued, this problem persisted which made it necessary to change how the exams were delivered. The final exam was delivered completely online. In the spring 2013 course, all exams were delivered this way.

Handling course communications. The original plan for handling administrivia relied on one graduate learning assistant to filter course communications and emails as soon as the semester began. Personal matters in the life of the graduate learning assistant prevented her from fulfilling this role in the first two weeks of the term. As a result, the instructor handled all course communications and determined that this role should always remain with the instructor. These communications provided an important connection with the enrolled students that might not have otherwise been made. This structural change remained in place, and the instructor now handles all email communications.

Sustainability

Will the redesign be sustained now that the grant period is over?

Despite the best efforts of the redesign team and lead instructor to communicate the goals and structure of the redesign, institutional changes in leadership have posed a number of obstacles. The department chair left UMSL just before the full implementation and the dean of the college of business administration is stepping down. As the information systems department shrinks in faculty FTEs and students and our institution faces substantial budget cuts in the near future, the challenge is to lobby for the necessary resources (a large teaching space to accommodate all students at one time) to fully embrace the principles of the redesign. The sustainability of the redesign currently rests on one instructor until the redesign team can reassemble with new leadership. The redesign has enjoyed support from the Center for Teaching and Learning and the Faculty Resource Center, and regular meetings with them will focus on building an institutional memory of the team’s original goals and experiences.

 

 

Quick Links:

Missouri Main Page